As personal computing devices become more powerful, containing increased storage space and processing capabilities, the average user consumes an increasingly smaller percentage of those resources in performing everyday tasks. Thus, many of today's personal computing devices are often not used to their full potential because their computing abilities greatly exceed the demands most users place upon them. An increasingly popular method of deriving use and value from the unused resources of powerful modem personal computing devices is a distributed computing system, in which the computing devices act in coordination with one another to provide more reliable access to data and computational resources.
In addition to providing a useful mechanism for using excess computing capacity, distributed systems can also be composed of dedicated inexpensive computing devices in order to achieve the performance and storage capabilities of a larger, more-expensive computing device. A further advantage of distributed systems is the ability to continue to operate in the face of physical difficulties that would cripple a single, larger computing device. Such difficulties could include: sustained power outages, inclement weather, flooding, terrorist activity, and the like.
To compensate for the increased risk that individual member computing devices may become disconnected from the network, turned off, suffer a system malfunction, or otherwise become unusable, redundancy can be used to allow the distributed computing system to remain operational. Thus, the information stored on any one personal computing device can be redundantly stored on at least one additional personal computing device, allowing the information to remain accessible, even if one of the personal computing devices fails.
A distributed computing system can practice complete redundancy, in which every device within the system performs identical tasks and stores identical information. Such a system can allow users to continue to perform useful operations even if all but one of the devices should fail. Alternatively, such a system can be used to allow multiple copies of the same information to be distributed throughout a geographic region. For example, a multi-national corporation can establish a world-wide distributed computing system.
However, distributed computing systems can be difficult to maintain due to the complexity of properly synchronizing the individual devices that comprise the system. Because time-keeping across individual processes can be difficult at best, a state machine approach is often used to coordinate activity among the individual devices. A state machine can be described by a set of states, a set of commands, a set of responses, and client commands that link each response/state pair to each command/state pair. A state machine can execute a command by changing its state and producing a response. Thus, a state machine can be completely described by its current state and the action it is about to perform, removing the need to use precise time-keeping.
The current state of a state machine is, therefore, dependent upon its previous state, the commands performed since then, and the order in which those commands were performed. To maintain synchronization between two or more state machines, a common initial state can be established, and each state machine can, beginning with the initial state, execute the identical commands in the identical order. Therefore, to synchronize one state machine to another, a determination of the commands performed by the other state machine needs to be made. The problem of synchronization, therefore, becomes a problem of determining the order of the commands performed, or, more specifically, determining the particular command performed for a given step.
One mechanism for determining which command is to be performed for a given step is known as the Paxos algorithm. In the Paxos algorithm, any of the individual devices can act as a leader and seek to propose a given client command for execution by every device in the system. Every such proposal can be sent with a proposal number to more easily track the proposals. Such proposal numbers need not bear any relation to the particular step for which the devices are attempting to agree upon a command to perform. Initially, the leader can suggest a proposal number for a proposal the leader intends to submit. Each of the remaining devices can then respond to the leader's suggestion of a proposal number with an indication of the last proposal they voted for, or an indication that they have not voted for any proposals. If, through the various responses, the leader does not learn of any other proposals that were voted for by the devices, the leader can propose that a given client command be executed by the devices, using the proposal number suggested in the earlier message. Each device can, at that stage, determine whether to vote for the action or reject it. A device should only reject an action if it has responded to another leader's suggestion of a higher proposal number. If a sufficient number of devices, known as a quorum, vote for the proposal, the proposed action is said to have been agreed upon, and each device performs the action and can transmit the results. In such a manner, each of the devices can perform actions in the same order, maintaining the same state among all of the devices.
Generally, the Paxos algorithm can be thought of in two phases, with an initial phase that allows a leader to learn of prior proposals that were voted on by the devices, as described above, and a second phase in which the leader can propose client commands for execution. Once the leader has learned of prior proposals, it need not repeat the first phase. Instead, the leader can continually repeat the second phase, proposing a series of client commands that can be executed by the distributed computing system in multiple steps. In such a manner, while each client command performed by the distributed computing system for each step can be thought of as one instance of the Paxos algorithm, the leader need not wait for the devices to vote on a proposed client command for a given step before proposing another client command for the next step.
A variant of the Paxos algorithm, known as the fast Paxos algorithm, streamlines the repetition of the second phase by removing the leader and essentially treating requests received directly from clients as requests that were sent by a leader. Each device votes on a requested function, and if a quorum of devices has voted for a function, that function can be executed and the request can be responded to. Because a leader no longer exists to order the clients' requests, the fast Paxos algorithm can function properly if each of the devices receive the same requests in the same order. However, if some devices do not receive the requests in the same order as other devices, a quorum may not select a function and a conflict can occur. In such a case, the Paxos algorithm can be restarted, and the leader can learn of the previous votes and submit one or more of the conflicted functions for a vote in an ordered manner. Additionally, because the Paxos algorithm relies on a leader to select proper functions to be voted upon, and the fast Paxos algorithm can operate without a leader, a quorum of devices for purposes of the fast Paxos algorithm can be defined as a larger grouping of devices to provide mechanisms by which a selected function can be determined if some devices fail.
The Paxos algorithm can provide a response to a client's request in as few as four message delays. Specifically, the transmission of the request to a leader results in one message delay; the transmission of the proposal from the leader to the devices results in a second message delay; the transmission of the votes from the devices to the leader results in a third message delay; and the transmission of the response from the leader to the client results in a fourth message delay. By removing the leader, the fast Paxos algorithm can provide a response to a client's request in as few as three messages delays. Specifically, the transmission of a client's request to the devices results in one message delay; the transmission of the devices' votes to a learner, or to each other, results in a second message delay; and the transmission of a response from the learner, or from the devices, to the client results in a third message delay. Because a distributed computing system may be composed of computing devices that are physically not close to one another, message propagation delays, even in an efficient network environment can dwarf other processing. Consequently, the message delays introduced by any consensus algorithm can act as a limiting factor on the efficiency and speed of that algorithm.
Therefore, what is needed is a distributed computing system that can use a consensus algorithm with fewer message delays, such as fast Paxos, but that can still recover from a conflict in an efficient manner.